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A method and system for automatically connecting real-world entitjes 
directly to corresponding network-based data sources or services 

FTFXD AND BACKGROUND OF THE I NVENTION 

BACKGROUND OF THE nvTVENTION 

1. Field of the Invention 

The present invention relates to an improved method and system for searching 
for and interacting with network-based information sources or any connectable 
information sources (CIS) and services, by usmg real-world elements as links to that 
information, or as triggers for system actions; 

2. Description of the Related Art 

One of the primary functions that the Internet and other connectable info 
sources enable is the provision of massive, varied, global information sources and 
services. Typical means of connecting users to such information sources and services, 
for purposes such as researching subject matter, executing transactions or contacting 
companies/individuals etc. entail connecting the user of an Internet compatible device 
to a specific Web site or page where ihe relevant information is found. This is usually 
initiated by typing an address, cUcking on a hyperlink or using a search engine for 
search purposes. In order to find relevant information it is typically necessary for a 
user to use text-based means, such as typing m the name or keywords of an object 

Alternative means have also been developed to enable navigation and 
searching using voice recognition technology. The increasing use of the VoiceXML 
standard, which combines the Extensible Markup Language (XML) with advanced 
voice recognition, providing interactive access to the Internet via phone or voice 
browsers. FoUovsdng a collaboration of AT«feT, IBM, Lucent Technologies and 
Motorola, VoiceXML was adopted in March 2000as a way to "voice enable" Internet 
applications, by the World Wide Web Consortium standards group. Voice navigation 
systems enable navigation of elements or objects by speaking them, but this still does 
not enable the usage of the objects themselves in the searching procedure. 
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The search for improved information searching techniques has lead to the 
development of various technologies that enable the usage of the real world 
elements/objects themselves to activate the mformation searches. AirClick (5 Valley 
Square Park, Suite 200, 512 Township Line Road, Blue Bell, PA 19422, USA, 
http://www.airclic.com/), for example, can connect the user to a web site by scanning 
a bar code, such that a user is not requires to type in any data in order to initiate . an 
accurate search. Another company, WuliWeb Inc. (1265 Birchwood Drive, 
Sunnyvale, CA, 94089-2206, USA - www.WuliWeb.com") enables the user to type tihe 
numbers that are printed above a bar code and than be connected to the relevant web 
page. These technologies, however, are limited in their applicability to bar-coded 
objects. 

There is thus a widely recognised need for, and it would be highly 
advantageous to have, a system and method that can enable the automatic lioking of a 
variety of real world elements and objects to online information sources and services 
for the purposes of research, commimication, security or commerce. 

SUMMARY OF INVENTION 

According to the present invention there is provided a system for enabling the 
use of real-world objects or elements (including data/information segments) as direct 
links (Iiyperlinks) to network based information, services or commercial sources. 

Specifically, the present invention enables a network (including Internet or 
altemative connectable information source (hereinafter referred to as "CIS")) enabled 
device with data-acquisition capabilities (including camera, scanner, sound recorder, 
smeller device, sensor etc.), to connect real-world elements or objects directly to 
corresponding Web sites, CIS or services related to the objects. The connection is 
initiated, either by the user or is automatically triggered by the device. 

The following expressions, referred to hereinafter, include the following 
classifications: 

CIS Any "Connectable Information Source", such as the Internet, 

intranets, extranets, the World Wide Web and dedicated networks. 

Network: A system that transmits any combination of voice, video and/or 

altemative data between users. This includes the Internet, Intranets, Extranets 
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and all other data networks, wherein data is shared, stored, queried, processed 

or transferred between network elements. 
Network elements: Include databases, routers, servers, switches, bridges, client 

devices, host devices etc. 
Network Server: A server that includes functions of Web servers. Intranet 

servers, network access servers and any other CIS that enable information 

processing and client requests to be processed and served. 
Network enabled: Any device or machine that has a communications component 

enabling connectivity to a data network, such that the device can communicate 

data to and from the network. 
Real World Elements: Any objects or data segments that may be sensed by humans or 

alternative sensor mechanisms. 

The present invention is comprised of: 

i. At least one network enabled device for capturing real world objecf s data 
and communicating with a network; 

ii. Device (Client) software for processing and enabling interacting with the 
object's data; 

iii. A network server system for processing requests from the network enabled 
devices and other network elements; and 

iv. Any kind of information, data, or knowledge database for storing links to 
information sources or services, or actual information or services. 

The process according to which the present invention operates, comprises the 
steps of: 

i. Capturing data from the real world - by taking a sample in, using a (client) 
network-enabled device; 

ii. Optionally, initial processing of that data within the device; 

iii. Connecting the user device to a network server or dedicated server, in order 
to enable matching up of the object's data, representation or description to a . related 
information or service source; and 
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iv. Transferring the object related data or service to the device, for viewing, 
hearing, , sensing, buying or otherwise utilizing the information. 
V. Optionaiiy, initiating an action, such as a request, emergency call, 

telephone call, transaction, alerting the user etc. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The invention is herein described, by way of example only, with reference to 
the accompanying drawings, wherein: 

FIGURE 1 is an illustration of the components and basic operations according 
to the present invention. 

FIGURE 2 illustrates an example of a cellular phone graphical user interface. 

DESCRIPTION OF THE PREFERRED EMBODIMENT 

The present invention, hereinafter referred to as the "object connector 
system", relates to a system and method for enabling the use of real-world objects or 
elements (data .segments (such as an object's bitmap, pieces of music) or information 
segments (such as electromagnetic radiation - Radio broadcast)) as direct links (such 
as hyperlinks) to information, knowledge, service provider and data sources (such as 
the Internet, World Wide Web, extranets, service centers etc.). 

The following description is presented to enable one of ordinary skill in the art 
to make and use the invention as provided in the context of a particular application 
and its requirements. Various modifications to the preferred embodiment will be 
apparent to those with skill in the art, and the general principles deJBned herein may be 
applied to other embodiments. Therefore, the present invention is not intended to be 
limited to the particular embodiments shown and described, but is to be accorded the 
widest scope consistent with the principles and novel features herein disclosed. 

Specifically, the present invention enables an Internet or CIS enabled device 
(wireless/cellular phone, NetPhone, PDA, portable computer, pager, computer, digital 
camera etc.) with data-acquisition capabilities (such as a camera, scanner, sound 
recorder, smeller device, probe, etc.), optionally computational capability (CPU, 
software), a connection (wireless or wireline) to information (such as the Internet, 
telephone directory) and a Man-Machine interface (MMO for providing an interface 
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to enable connecting real-world objects directly to their corresponding network-based 
information or service sites. 

An example of such as system is an Internet enabled cellular phone equipped 
with a camera. The user can point the camera at an object, take a photograph of the 
object, and then press a key to send this bitmap image to a server for further 
processing/research. Alternatively, the device itself may undertake initial processing 
of the object data, and than send the result of the processing to the Web server. The 
user is then connected to a Web page (or a list of hyperlinks) which includes that 
specific photograph or relevant information about it. 

The object connector system is enabled to capture data from any source, and 
make use of the data according to its specific type, such as searching of dedicated 
sound, taste, smell, audio or graphic-based databases, including the following: 

SEEING images, graphics, movement, video 

HEARING soimds, mxisic, voices 

SMELLING smells 

FEELING feel, touch 

TASTING tastes 

SENSING waves, energy, forces, time 
The object connector system can also capture information that our senses cannot 
capture, such as: 

Eiectromagnetic radiation, ultrasound, radio waves, slow changes (movement 
of clock hands), vibrations (pre-earthquake), low-heat sources and undersound (sound 
at frequencies which are lower than human hearing capability: < 1 8 Hz). These 
various information sources may be utilized by the object connector system, by 
employing a communications/computing device with an appropriate input mechanism 
for the relevant information source. Subsequently, the information is captured, 
optionally processed, and fransferred via the device to a network server for further 
research etc. 



The object connector system can be integrated into a device (e.g. cellular 
phone, PDA) that is network-enabled and that incorporates a sensor (e.g. camera. 
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microphone etc.). Any other mtemet-enabled devices with any kind of sensor can also 
be utilized for the purpose of the present invention. 

Txie object connector system captures information (data) from the real world 
(e.g. an image) using the sensor (e.g. camera), and subsequently links to a database 
(or search engine in order to find a reference to this data on a relevant network based 
source. For example, an image based search engine can screen a visual database to 
find tijas SpcuirLc Image, iiubjecL, item or service on a particular vveb page. 

The object connector system of the present invention can optionally perform 
some analysis on the object or image ^vithin the device itself, such as capturing text, 
perforaiing Optical Character Recognition (OCR). and using this added information to 
enable more accurate searching (e.g. identify the web address that appears on an 
advertisement as added inforrnation), OCR is well known in the art and is in 
commonly used in online dictionaries (such as Babylon, from Babylon Ltd., 10 
Hataasiya Street, Or- Yehuda, Israel, 60212), offline dictionaries (such as 
quicktionary, from Quick-Pen.com, from Kansas City, MO 64145-1247), scaimers 
etc. 

The object connector system can optionally use extra information that the 
cellular phone system can provide, such as geographical location by triangulation or 
by GPS to locate the cellular phone, and incorporate it into the otiier data acquired. 
The object connector system can also use extra information firom other sources, such 
as temperature, humidity and movement, and perform data-fusion to support and 
focus the basic data segment to enhance its relevancy in locating the relevant web 
page, database or service. It is possible for a user to predefine conditions or rules 
(such as, when I arrive in city X, remind me to visit aunt Sara) that vsdll act as triggers 
for device's actions. In this case, the device acts together with information passively 
received (such as location, weather, moods, smells, sounds etc.) to initiate a pre- 
configured request or alert. 

An additional example of the application of the object connector system is the 
case where the user aims his or her phone at a hoteVs name (Logo), captures it and 
automatically gets connected to the hotel chain's reservation office. The cellular 
phone system can automatically add the user's actual location information (e.g. Sth av. 
on 55th St.), and the hotel's reservation system vwU send the user the specific 
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information about the actual hotel that he or she is now looking at. The device (for 
example a cellular phone) can subsequently be connected to the actual Web page of 
this hotel reception-desk. The user can then look at the information, study it, analyze 
it and decide what to do with it (choose a room, request for more information, check 
special offers, make an order, etc.). 

The object connector system can then connect the device (for example a 
cellular phone) to a Web site (or a list of hyperlinks to information sources, search 
tools etc.) and display the relevant infonnation to the user (on a screen, vocally etc.). 
The user can then look at the information, study it, analyze it and decide what to do 
with it (such as buy an item, get store information, go there etc.). 

Detailed description of the parts 

The present invention consists of: 

i. At least one network enabled device 10 for capturing real world object data 
and communicating with a network; 

ii. Client software in said device, for enabling interacting with the object data 

and optionally processing the object data; 

iii. A network server system 12 for processmg requests firom the network 
enabled devices and other network-based elements; and 

iv. Any kind of information, data, or knowledge database 14 for storing links to 
network based information sources or services (such as databases, search engines 
and connections to service providers, including Police, security company, 
emergency services, etc.), or actual data sources. 

1 The device includes: 

i. At least one sensor, or data capturing mechanism (such as a camera, scanner, 
smeller mechanism, microphone, antenna, taster mechanism, feeler 
mechanism, IR sensor, geophone (which is an electronic receiver designed to 
pick up seismic vibrations), radiation meter, movement meter, acceleration 
meter, wind meter, thermometer, humidity sensor etc. 
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ii. A communications mechanism for enabling data transfer betvv^een the device and a 
network, including wireless/wireline access to the Internet, Intranet or other 
information souious). 

2. The device's fclient^ software includes: 

i. Man-Machine Interface (MMI), providing features such as menus, emergency 
buttons, audio interaction, voice leco^uiliuju (Lo choose menu items, etc.) for enablmg 
user interaction with tiie data; 

ii. Optionally data processing and storage capabilities (image capture, image 
compaction, OCR, etc.). These capabilities enable the data to be captured and 
optionally processed and stored. Such capabilities enable, for example, the device to 
optionally execute additional processing of the object data, such as filtering the data 
(for example, discerning a URL on an advertisement) or adding relevant alternative 
factors (for example, the users current geographic location); 

iii. Optionally, a local engine/database to undertake local research on the captured 
object, so as to maximize search accuracy and efficiency. This processing engine can 
search within the device in: 

1 User preference lists, or instruction lists which are stored in the 
device's memory; 

2 The device's memory for previous searches; and 

3 Other devices memory options, such as telephone lists, events, 
documents, photos. 

The client software enables the user to: 

i. Enable capturing of the data, through the sensor or sensors. 

ii. Optionally, to add instructions and extra information (i.e. tell the search engine to 
look for the data in a specialized database, such as searching for an image in the 
logos database by typing or saying "logo"). 

iii. Process the data and incorporate relevant factors such as weather conditions, 
timing, geography, topography, events, history, user's mood, user's vital parameters 
(such as heartbeat, breathing, temperature). This processing may additionally 
incorporate relevant items from the memory of the device. 
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iv. send raw or processed data to a remote information center (web server etc.) 
V. store information for later use. 

It is noted that the location of the software and hardware modules of the object 
connecting system can differ from one hnplementation to another. 

3. The Network server system i ncludes: 

i. A communications center for receiving and servmg data to and from system users; 

and 

ii. a processing component for processing and serving requests. 

4. The data source includes: 

i. At least one database for storing links to object-related data, or the data itself. This 
database may optionally include at least one speciaUzed search engine (e.g. 
Image/smell/taste/sound/feeling based searches using pattern matcliing) for enabling 
searches of network-based data related to captured object This a user may be Hnked 
to data found in sources such as Web sites, intranet sites, extranet sites, databases, 
search engines, and service centers, or may access the data directly from the primary 
data source. The database may include fields such as: 

a] Web site links to links to other sites or information sources; 

b] Other optional databases, including: Image, sound, smell, feel, speech based 
databases; and 

c] User preferences and details, client responses, security codes etc. 

d] Other means of connection (telephony, wireless etc.). 

In an alternative embodiment, the device can connect itself directly to the 
target (e.g. a web page) by self-performing some processing (OCR) that generates an 
address (URL). 

Detailed description of the proces s according to the present 
invention 

The "object connector system" achieves this connection in using the foDowing 

steps: 
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1 . Capture data from the real world - take a sample in using a client network-enabled 
device, either through initiation by the user, or through an automated process by the 
device itself. 

2. Optionally, initial processmg of that data within the device. 

3 . Connecting the chent device to a network server or a dedicated server (may include 
Web server. Intranet server. Service provider server etc.), via a data network, in order 
to uiitioli ilic objtscL's daiarcprcscuuiuou oi Jcsciipliuu wiih rclaicu Jala suluCcs ui 
services online. 

4. Get the object-related data or service from the relevant online source, and transfer it 
to the device for viewing, buying or otherwise utilizing. After the connection is 
achieved, interactive searching is enabled, including studying, analyzing, seeing, 
hearing, smelling, feeling and ta.'rting of the object-related data. 

The getting of this data optionally includes accessing and interaction with this 
data from the (client) device itself 

5. Optionally, automatic or user initiating of at least one pre-configured action, such 
as an emergency call, aler^ transaction, alarm etc. 

As can be seen in Figure 1: 

i. A client device 10 is instructed to view/hear/smell/touches/sense/feel 21 a real 
world object 16, and subsequently to choose or capture the real world object data. 
Alternatively, the device may be configured to automatic receive the data without user 
initiation, such as receiving geographic data based on the device's current location. 

ii. The data of the object 16 is captured 22 and optionally processed by the device 10. 

iii. If not processed by the device 10, the device 10 sends 23 the object data or 
processed data to the Network server 12, in the form of a request The request is sent 
via the Internet 18 or any other data network. 

Alternatively, the device can alternatively connect itself directly 30 to an 
external (dedicated) information source, as in the case where some processing occurs 
in the device (such as OCR online), or if the link ahready exists in the device memory. 
In these cases, the device 10 sends 32 the data to a dedicated server 31, via a 
dedicated connection. An example of this is a security company that has placed 
dedicated "red buttons" (for emergency alerts) on client devices. Upon pressing the 
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button, a Bser may be connected directly to the dedicated server of the company, 
powered by the object connector system, which wiU serve the request. 

iv. The Network Server 12 receives the request 24 from the Internet 18 and queries 25 
the relevant local database/information source 14 for appropriate information or links. 
If the required information is found in this local data source, ihc information is sent 
back to the device 28. 

v. If a request requires linking to a network 18 (such as the World Wide Web) or 
another external data source or service provider, the device 10 or the database 14 
sends a request 23, 26 to the network-based 18, mformation source or service 
provider, such as a Web site, search en^e, or to a dedicated information or service 
provider, via a dedicated server 31. 

vi. The information or service source 18 responds to the request, sending 27 the data 
to the server 12 or directly to the device 23. In the case where the information request 
was processed by the dedicated server 31, the response is similarly sent either back to 
the Network Server 12 or directly to the Device 32. 

vii. In the case where Ihe data is sent to the Server 12, the server 12 subsequently 
sends 28 the data to the device 10. 

viii. The device 10 receives the data, and the user subsequently 
reads/smells/viewsAistens to/tastes/feels the data. The user can thereby surf the CISs 
and initiate subsequent requests at wiD. 

The above method can be described as follows: 

The aim of the present invention is to transform a piece of raw sensor data 
(such as bitmap data) into a database (DB) address (i.e. a URL) or to initiate an 
action (alann, reminder), ha this example, there are at least three ways of executing 
the process: 

1 . One on one match (using pattern matching (pattern recognition)) of the data 
(i.e. bitmap) to the data base data, and from there to extract the address; or 

2. Extract a minimal amount of data from the bitmap that suffices to identify the 
image in order to estabUsh an address, referred to as minimizmg. This may entail a 
process of reducing resolution of the image in order to minimize data transfer, while 
retaining enough clarity in order to create a viable pointer (until it is the smallest. 
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viable pointer). For example, if the database contains 100 information objects, than a 
bitmap resolntion of 1 0 X 1 0 may suffice, in order to establish a viable pointer to at 
least one one of the above mentioned information objects. 

3. Introduce data-fusion techniques to incorporate additional information to the 
sensor data, such as performing Optical Character Recognition (OCR) on a newspaper 
advertisement. This process thereby focuses the match of the sensed data to the 
iclcvaiul uaLabiise, in order to identify a URL address in a more specialized aresL An 
example is using GPS technology or using the cellular service provider's information 
about the device's location, such that a geographical component is added to the 
captured image, and the subsequent matching of the object to the database and tiie 
URL link incorporates the geographical limitation. 

An example of a graphic user interface according to the present invention can 
be seen with reference to Figure 2: As can be seen in the figure, the graphic user 
interface of the device may present the user with relevant search options, such as 
menu 50 with options to learn more or browse 52, save the data 54 or buy 56 the 
captured object 58. The menu 50 may be customized according to the type of data 
able to be captured. For example, a device with a smeller mechanism (sniffer) may 
provide options to learn, smell, mix and buy. 

In a preferred embodiment of the present invention, there is provided a 
network enabled device v^th an integrated camera (or scanner), such as: cellular 
telephone, NetPhone, PDA, Portable computer, personal computer, pager, Internet 
enabled appliance, gadget or machine. The present device is constructed using 
existing components such as mobile devices with scanning means, smelling means, 
picture/video capture means, audio capture means, touch sensitive means and taste 
sensitive means. 

In an additional embodiment of the present invention, the object data captured 
or utilized by the client device can be stored for later use, such as studying it later or 
transferring it to another device (a PC, PDA, computerized-refrigerator, etc.). 

In a still further embodiment of the present invention, the client software 
enables an application that automatically alerts the user, based on geographical, 
topographical, time-related, and situation related factors. For example, the device that 
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captured the object data can be configured to automatically respond to certain eventS;, 
such as send a warning signal to the user when sensing higher than average radiation, 
alerting the user to unusual climate or odors, sending the user alerts based on 
geographical location etc. These actions or events may be pre-stored in the device 
memory or in a remote database, accessible to the device. 

Example 1: A person aims his or her digital camera (with network connectivity 
facility) at an object (car, printed advertisement) and takes its photograph. The camera 
captures the image and displays it on the screen: The user chooses a part or all of the 
image, presses a button and gets connected to a network server that connects the user 
device to a relevant database. This database either answers the request, or refers the 
request to an external database, Web site or search engine, that searches the web for 
this specific image (using pattern matching, minimizing, reducing resolution and data- 
fusion, etc.). Once the user is connected to an information source, such as a Web page 
or a list of hyperliiiks, he or she can navigate there, study the infomiation and get 
connected to other relevant sources. 

The user can then xise all the Internet facilities such as e-Commerce, 
navigational information, purchasing and reservation systems etc. The user can also 
compare prices, contact dealers and purchase the object that he or she saw. 

Example 2: The user can point a cellular telephone device, which is powered with 
the client software of the present invention, at an object, take a digital photograph of 
the object, and inmiediately be connected to a corresponding Web page that includes 
the captured photograph and/or information about the photograph. Such a cellular 
telephone is an Internet enabled cellular telephone equipped with a digital camera, 
which enables the capture and usage of real world objects (such as an image of a 
flower or an advertisement) or data segments (such as pieces of music) or information 
segments (such as electromagnetic radiation, radio broadcasts) as direct links (such as 
hyperlinks) to information, knowledge and data sources (such as the Web, Internet, 
extranets, intranets etc.). The user can subsequently execute further research, initiate 
transactions, process requests, or altematively store the image and information for 
later use. 
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Example 3: The user can aim his or her cellular phone (with a camera ftinction) at 
an adverdsenient blllbuard nedi Oie lilguway, caplurc Ihc picture and gel cuimecled to 
the relevant dealer or web page (using location information that is acquired from the 
cellular service provider). 

Exajspic 4: In an emergency situation (rabbeiy etc.), the asei pushes a chosen 
button (the "red button") on his or her cellular phone, and: 

i. A picture is taken of the offender; 

ii. The phone connects to an emergency call center (police) and sends the bit map 
image and the geographical location of the incident, and continually transfers voice 
and photographs to this center. 



Example 5: Accident sensor that responds to accident parameters (shock, noise, 
rotation) and automatically contacts an emergency center. 

Example 6: Outdoor personal alarm (JR, volume, movement sensor) that alarms the 
user about an approaching intruder. 

Example 7: An improved personal "emergency button" for asthmatics (Keeps in its 
memory typical asthmatic sounds and responds by contacting emergency services or 
automatically initiating a reminder to tihe user upon identifying such sounds; and heart 
patients (monitor relevant parameter/s and respond accordingly). 



Example 8: A military or a security services provider device, such as device for 
guards or soldiers, wherein: 

i. Tlie guard clicks upon arrival at predefined station to monitor his/her job 
performance. Each click sends a signal (photo, geographic location) to th 
company's control center to monitor the guard* s performance. 
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ii. In case of emergency (an intruder), the guard presses a "red button" that sends 
an alarm, a photo of the intruder, guard voice and a sound recording to the 
control center. The device continues data transfer thereafter. 

iii. Optional: A virtual Guard: 

The Guard (soldier) leaves the device in a particular place. The device is 
programmed to respond to predefined signals and to send the data back to the 
center. 

The foregoing description of the embodiments of the invention has been 
presented for the purposes of illustration and description. It is not intended to be 
exhaustive or to limit the invention to the precise form disclosed. It should be 
appreciated that many modifications and variations are possible in light of the above 
teaching. It is intended that the scope of the invention be limited not by this detailed 
description, but rather by the claims appended hereto. 



